Jointly developed by and with versions maintained by the ISO/IEC and ITU-T standards organizations, H.264, a.k.a. Advanced Video Coding (AVC) and MPEG-4, Part 10, is a commonly used video coding standard that was designed in consideration of the growing need for higher compression of moving pictures for various applications such as, but not limited to, digital storage media, television broadcasting, Internet streaming and real-time audiovisual communication. H.264 was designed to enable the use of a coded video representation in a flexible manner for a wide variety of network environments. H.264 was further designed to be generic in the sense that it serves a wide range of applications, bit rates, resolutions, qualities and services.
The use of H.264 allows motion video to be manipulated as a form of computer data and to be stored on various storage media, transmitted and received over existing and future networks and distributed on existing and future broadcasting channels. In the course of creating H.264, requirements from a wide variety of applications and associated algorithmic elements were integrated into a single syntax, facilitating video data interchange among different applications.
Compared with previous coding standards MPEG2, H.263, H.263+ and MPEG-4, H.264/AVC possesses better coding efficiency over a wide range of bit rates by employing sophisticated features such as using a rich set of coding modes. In this regard, by introducing many new coding techniques, higher coding efficiency can be achieved. To achieve such a high coding efficiency, H.264 is equipped with a set of tools that enhance prediction of content at the cost of additional computational complexity.
A macro-block (MB) is a term used in video compression as a standardized sub-size of image frames, i.e., image frames can be broken into and processed as macro-blocks. In H.264, macro-blocks are used, which each represent a block of 16 by 16 pixels. In the YUV color space model, each macro-block contains 4 8×8 luminance sub-blocks (or Y blocks), 1 U block, and 1 V block (represented 4:2:0, wherein the U and V blocks provide color information). Color can also be modeled or represented by the 4:2:2 or 4:4:4 YCbCr format (Cb and Cr are the blue and red Chrominance components).
Most video systems, such as H.261/3/4 and MPEG-1/2/4, exploit the spatial, temporal, and statistical redundancies in the source video. Some macro-blocks belong to more advanced macro-block types, such as skipped and non-skipped macro-blocks. In non-skipped macro-blocks, the encoder determines whether each of 8×8 luminance sub-blocks and 4×4 chrominance sub-block of a macro-block is to be encoded, giving the different number of encoded sub-blocks at each macro-block encoding times. It has been found that the correlation of bits between consecutive frames is high. Since the level of redundancy changes from frame to frame, the number of bits per frame is variable, even if the same quantization parameters are used for all frames.
Therefore, a buffer is typically employed to smooth out the variable video output rate and provide a constant video output rate. Rate control is used to prevent the buffer from over-flowing (resulting in frame skipping) and/or under-flowing (resulting in low channel utilization) in order to achieve reasonable video quality. For real-time video communication such as video conferencing, proper rate control is more challenging as the rate control is employed to satisfy the low-delay constraints, especially in low bit rate channels.
Scalable Video Coding has been attracting attention due to its high flexibility. The current H.264/AVC scalable extension has adopted the Motion Compensated Temporal Filter (MCTF) framework to provide temporal scalability. However, faster motion estimation (ME) algorithms are desired because ME can introduce unnecessary delays based on the MCTF framework. In short, the reference software provided with the MCTF provides an inefficient solution for motion compensation.
Accordingly, it would be desirable to provide improved motion estimation for scalable video coding. The above-described deficiencies of current designs for video encoding are merely intended to provide an overview of some of the problems of today's designs, and are not intended to be exhaustive. Other problems with the state of the art and corresponding benefits of the invention may become further apparent upon review of the following description of various non-limiting embodiments of the invention.